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ABSTRACT 

This article presents a problem-based learning (PEL) model for teaching a college US. history sun'ey course (U.S. 
history since 1890) designed to promote postformal thinking skills and identify and explain thinking systems 
inherent in adult complex problem-solving. We also present the results of a study in which the outcomes of the PEL 
model were compared to the outcomes of the same course taught with traditional lecture and discussion. The PEL 
model was more effective in scaffolding learning so that students recognize and practice postformal thinking 
dynamics and in facilitating self-reported student perceptions of increased course engagement and content 
relevance. We offer recommendations for implementing PEL in social science survey courses. 
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INTRODUCTION 

uch of the recent scholarship on teaching and learning (SoXL) of history has focused on moving 
away from a content coverage model in survey courses toward instructional methods that engage 
students in the practice and cognitive habits of the historian's craft (Andrews, 2009; Calder, 2006; 
Coventry et aL 2006; Jeffry, 2003; Steams. 1993; Sipress & Veolker, 2011). Calder (2006) described this signature 
pedagogy as one that requires students to “do. think, and value what practitioners in the field are doing, thinking, 
and valuing” (p. 1361). Sipress and Voelker (2011) referred to this recent instructional tlirust as a “pedagogical 
countennovement” (p. 1051), in which the development of historical thinking skills becomes the primary- goal of the 
surv ey course. Westhoff (2013) described historical thinking as the second of three fronts that created “a perfect 
storm” (p. 3), which helped to reshape U.S. survey courses over the last 2 decades; the Internet and the release of the 
1994 U.S. History Standards were the other two fronts. This reshaping has resulted in historical thinking and related 
cognitive skills, becoming a central focus of many U.S history survey courses at both the secondary and college 
lev els. VanSledright (2014) offered a model of learning history that identified historical thinking as the strategic 
knowledge necessary to do history and engage in domain specific cognitive efforts, which include “searching out 
evidence, assessing the sources for what they may offer, making sense of the perspectives of the sources’ authors, 
judging an author's reliability- in making reputable claims—in sen ice of answering historical questions of interest” 

(p. 6). 

The U.S. history survey course plays a key role in most general education curricula, by helping students achieve 
outcomes often framed around critical thinking and participatory citizenship. Laville (2011) described how a 
historical thinking-based pedagogical approach aligns well with these outcomes: 

Such an approach encourages the development of those intellectual and affective qualities that students will 
need in the exercise of their civil responsibilities. It calls upon the critical faculties common to historians - 
their ability to isolate a problem, analyze its component parts, and offer an interpretation - as well as their 
qualities of curiosity , empathy, and so forth, all of them built upon a solid foundation of analytical 
reasoning, (p. 166) 
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Wynn (2015), however, questioned the sustainability of historical thinking skills practiced in a U.S. history survey: 

Assuming these students develop historical thinking to some competent degree over a semester or two. the 
question arises to what extent they will be able to maintain and apply domain-specific knowledge and 
reasoning skills beyond the context of the survey course? (p. 29) 

Bain (2009) posed a similar question: “I have no idea what happens when students move into other settings. Does 
any of this have staying power, or is it merely contextualized to ‘that is how we studied history in our freshman 
year?’” (p. 166). 

Laville (2011) addressed the extent to which historical thinking skills will be useful to students as they confront 
broader, real world problems and issues beyond their survey course experiences: 

We must not forget that these pupils we are preparing in class will be less likely, as adults, to deal with 
history texts than to use their skills as citizens in the identification of social problems, the analysis of areas 
of conflict, the rational calculations of risks and rewards, and the weighing of competing interests, and in 
their personal decisions on the issues of the day. (pp. 166-167) 

Examining the multi-systematic nature of advanced problem-so king helps to define the broader cognitive dynamics 
necessary to successfully confront complex problems and issues. Sinnott and Johnson (1996) argued for problem- 
based university curricula that promote the development of these broader, more advanced cognitive skills among 
students, e.g. those found in postformal thinking operations that involve multiple thinking systems. The Association 
of American of Colleges and Universities (AAC&U; 2015) established principles and guidelines for developing 
general education curricula and experiences that guide students to “demonstrable, portable proficiencies aligned to 
widely valued areas of twenty-first century knowledge and skill,” including the development of sustainable problem¬ 
solving skills '“through problem-centered work on significant issues relevant to their interests and aims” (p. 3). We 
argue that the college and university U.S. history survey courses can be pedagogically and epistemologically 
structured to promote these proficiencies and the inherent advanced thinking skills. 

In this article, we expand on previous research (Wynn, Mosholder, Sc Larsen, 2014) that indicated that PBL with 
explicit metacognitive reflection promotes the practice and acquisition of postformal cognitive skills among first- 
year learning community students and non-learning community students in a LT.S. history survey course context. 
First, we identify and explain specific targets of advanced cognition inherent in complex problem-solving. Second, 
we provide a detailed description of our PBL metacognitive reflection model through which these systems/skills are 
identified and practiced. Lastly, we present results from the current study that measured the effects of the 
instructional model on the development of postformal thinking skills among U.S. histoiy' survey students (pre/post¬ 
test differences). In this study we compare pretest-posttest differences between U.S. history survey students who 
were taught using the PBL method, and students who were taught using TLD, and we measure the model 1 s impact 
on student perceptions of course engagement and content relevance. 

Formal Thinking, Postformal Thinking, and Adult Complex Problem-Solving 

Our basic goal in confronting students with complex problem-solving experiences in the context of a U.S. history 
survey course is to guide them to break away from a more closed systems problem-solving approach. Closed 
systems problem-solving is very common for students who have attained formal thinking skills, or the ability' to 
apply logical operations to abstract problems (Inhelder Sc Piaget, 1958). Closed systems problem solvers often fail 
to recognize various relevant variables and important contextual aspects of the problem or issue at hand and 
presume a logical consistency within a single logical system (Sinnott, 1998). This often leads them to expect to 
produce a single right answer that closely aligns with solutions to similar problems they have encountered in the 
past, while assuming that this right answer will hold in all similar circumstances in the future (Wu Sc Chiou, 2008). 

Students may also rely upon intuitive thinking when problem solving through an “if it feels right, it's right” 
dynamic. This approach tends to be much easier and feels more natural than applying the logical operations 
associated with formal thinking (Epstein, Pacini, Denes-Raj, Sc Heier. 1996; Evans, 2008; Steinberg, 2007; 
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Witteman, van den Bercken, Claes. Sc Godoy, 2009; Wynn. 2010. 2015). Students must come to recognize the 
inadequacy of formal and intuitive thinking systems in solving complex problems in order to prompt a search for 
more adequate problem-solving systems (B as sec lies, 2005; Sinnott. 1998). These more adequate and advanced 
thinking systems are inherent in adult complex problem solving and are found in the postformal stage of reasoning 
(Arlin, 1984; Basseches, 1984: Chiou, 2008; Commons. Ross. Sc Miller. 2010; Commons Sc Ross, 2008; Labouvie- 
Vief, 1985; Labouvie-Vief, Adams. Hakim-Larson. Sc Hayden. 1983; Marchand, 2002; Perry, 1970; Reigel, 1975; 
Sinnott. 1989. 1998) which includes two substages. 

Relativistic thinking is the first sub-stage, described by Sinnott (1998) as the recognition that complex problems 
often can prompt many ideas about what is true and relevant, and that these ideas can be conflicting. Context and 
perspective also become relevant in this stage. What is believed to be true in one context or from one perspective 
might not be true from another (Chiou, 2008). Kahlbaugh Sc Kramer (1995) studied the development of relativistic 
thinking in late adolescents and young adults and concluded that people in these age groups ate faced with an 
increasing number of personal, social and perhaps political challenges. These challenges may prompt a cognitive 
adaptation toward a more relativistic cognitive dynamic. Because their fixed truths and dualistic categories no 
longer provide adequate answers, relativistic thinkers consider the multidimensional nature, multiple truths, and 
contextual characteristics of problems and issues as they seek solution alternatives. Relativistic thinkers also 
recognize that some problems and issues do not have acceptable or workable solutions (Chiou, 2008; Sinnott, 1998). 

Dialectical thinking is the second and most advanced sub-stage of postformal thinking and adult problem solving 
(Basseches, 1984. 1989. 2005; Ho, 2000; Ledbetter, 1986; Riegel, 1975; Sinnott, 1998). Moving beyond relativistic 
thinking, dialectical thinkers recognize that contradictory positions within a complex problem or issue are 
connected, and must be considered, to develop solutions. They search for and recognize gaps in the logic of 
opposing positions or perspectives, including inconsistencies and contradictions, and seek to understand how and 
why the positions are constructed and defended. These gaps prompt the utilization of a variety of cognitive systems 
(intuitive, formal, relativistic, dialectical), resulting in solution alternatives reflecting higher levels of understanding 
and cognition (Ho, 2000). In addition, dialectical thinkers recognize that any solution to, or stabilization of, a 
complex problem or issue will always be subject to changes and challenges because this is the nature of the tension- 
to resolution-to tension cycle dynamic of issues (Blouin Sc McKelvie, 2012). Vukman (2005) summarized the 
relativistic arid dialectical characteristics involved in postformal problem solving from the work of Kr amer (1989, 
1990) and Sinnott (1984, 1989): 

• Acceptance of relativism: recognition that personal perspective is only one of many potentially valid 
viewpoints on reality; 

• Understanding and acceptance of contradiction and the ability" to handle conflicting ideas and systems: 
the realization that contradiction and complexity may be inherent features of reality 7 ; tolerance is 
gained through an appreciation of the dialectic relationship between opposing systems; 

• Integration of sj^stems/frames of reference: a synthetic form of thinking that integrates several 
opposing systems into an abstract whole and actually transcends idiosyncratic perspectives (Kuhn, 
Cheney, Sc Weinstock, 2000). (p. 212) 

Guiding students to practice these broad, postformal problem-solving skills is a central focus of our instructional 
model, along w ith the purposeful application of content know ledge and the development and application of domain- 
specific cognitive skills like historical thinking. 

Th e Role of Domain-Specific Cognitive Skills within Adult Complex Problem-Solving 

The metacognitive dynamic described in Demetriou, Spanoudis, Sc Mouyr s (2011) overarching stage theory also 
offers a useful conceptual framew-ork that influenced our instructional model. This theory’ helped define and explain 
the multi-systematic nature of postformal thinking, as w^ell as the role of discipline specific cognitive skill sets (like 
historical thinking) within the broader framework of adult complex problem-solving, and emphasized the 
importance of capitalizing on developmental milestones for learning. 
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A student who has reached adult learning milestones would recognize that real-world problem solving may require 
solving a series of subproblems related to the overarching problem at hand, each of which may require domain- 
specific reasoning (e.g., historical thinking, mathematical estimations of relations) before a solution alternative can 
be selected. 

Students must be given the opportunity to think on and assemble complex hierarchical problems where 
hierarchies are embedded into each other. Working on problems of this kind may enable students to 
differentiate between the various cognitive systems and processes activated in problem solving. (Demetriou 
et al., 2011. p. 645) 

This metacognitive dynamic helped guide the development of the reflective component of our instructional model 
and helped us better understand the importance of guiding students to recognize how discipline-specific cognitive 
skill sets, like historical thinking, which clearly involves post formal thinking dynamics, may be purposefully 
identified, selected, and utilized in the broader context of real-world adult problem solving. 

Promoting Postfonnal Thinking: A Problem-Based Learning Instructional Framework 

The recognition that exposure to complex problems, along with a modeling and scaffolding process, guides students 
to move toward a more postfonnal cognitive framework (Basseches. 2005), helped us to recognize an 
epistemological alignment with PBL. PBL is based on authentic, real-world problems (Hmelo-Silver. 2004), and 
encourages students to use advanced thinking processes (Lenkauskaite & Mazeikiene, 2012). Originally used in 
medical education. PBL has been utilized in a variety of settings ranging from middle school through all levels of 
postsecondary education (Hmelo-Silver, 2004). PBL has been shown to increase content knowledge retention and to 
improve student ability to transfer problem-solving processes into both new and more complex circumstances 
(Blumberg, 2000; Cognition and Technology Group at Vanderbilt [CTVG], 1997: Maxwell, Bellisimo, & 
Mergendoller, 2001; M erg end oiler. Maxwell, & Bellisimo, 2006). Skillful PBL instructors model problem-solving 
systems and guide students through thinking processes. These behaviors provide the cognitive scaffolding or 
guidance necessary to promote advanced PBL skills, such as postfonnal operational problem-solving skills (Wynn, 
2010, 2015; Wynn et al., 2014),. The addition of peer-to-peer interaction can provide a collaborative learning 
environment that prompts more capable peers to take on a similar tutorial role (Hmelo-Silver, 2004). 

When using our model as intended, instructors will demonstrate to students how r to recognize, select, and apply a 
variety of cognitive systems based on the characteristics of the problem or issue at hand. These can include the 
intuitive, formal, relativistic, and dialectical systems previously described as well as domain-specific systems like 
historical thinking. Once this framework has been presented, the instructor guides students through a reflective 
process in which they judge which systems were more useful or successful, and why. This metacognitive reflection 
guides students to monitor and direct the processes of problem-solving, and helps them construct a cognitive self- 
awareness that is necessary in the development of postfonnal thinking and problem-solving operations (Hacker, 
1998: Hmelo-Silver, 2004; Wynn, 2010, 2015: Wynn et al., 2014), in contrast to what Bartos and Banks (2015) 
defined as an “osmosis model” (p. 36), in which advanced thinking skills are somehow absorbed through 
participation in lectures and class activities. 

The PBL procedures used in our study are outlined as follows, and were based on Wynn’s (2010, 2015; Wynn et al., 
2014) PBL instructional model. These activities may be accessed tlirough the link provided in Appendix A. 

Step 1 - Problem Development: Our primary focus during this phase w r as to peak student interest (create a 
need to know more), establish “stakeholdership,” and explicitly portray the problem/issue as 
multidimensional. We presented problems/issues through story-telling, video clips, data presentations, and 
readings (including primary sources). 

Step 2 - Initiation of PBL Events-Argumentation and Student Inquiry 7 : Our initial focus w r as to guide 
students to define the problem, identify its multidimensional or multi-truth characteristics, and identify the 
need for domain specific knowledge and processes (discipline specific skill applications, inquiry, etc.) and 
broad thinking systems (intuitive, formal, relativistic, and dialectical). We then initiated a decision- 


Copyright by author (s): CC-BY 


4 


The Clute Institute 



Journal of College Teaching & Learning - First Quarter 2016 


Volume 13, Number 1 


based/argumentation structure (Jonassen, 2012) to prompt students to generate arguments, and allow them 
to work to resolve conflicts and contradictions among competing positions. This was done primarily 
through historical simulations and current issue presentations. At the conclusion of the simulations and 
issue presentations, students identified what they learned about the problem/issue and the inherent 
contradictory or opposing positions, and identified what additional information they needed in order to 
develop solution alternatives. 

Step 3 - Problem Solution: Students generated solutions/decisions and examined their 'Tit.” then proposed 
the most appropriate one. and evaluated its historical or potential consequences. We then guided students to 
compare their solution'decision to the actual historical outcome and its legacy, and the impact of change. 
Solutions/ dec is ions on current issues were analyzed base on potential for success. We assigned a 
concluding opinion essay, and guided students through debriefing, which included a review 7 of the content, 
concepts, and skills encountered and used dining the problem solving cycle. Students then reflected on the 
types of thinking strategies they used, and the successes or failures of each, through the use of a 
metacognitive reflection questionnaire (Appendix B; Wynn. 2010; Wynn et al., 2014; Wynn. 2015). 

RESEARCH QUESTION AND HYPOTHESES 

Based on this literature review and previous research (Wynn et al.. 2014). we hypothesized that participation in 
multiple PBL activities that contained a metacognitive reflection element would have a significant relationship with 
the development of postformal thinking skills among U.S. history survey course students. W 7 e also hypothesized that 
PBL students would report an increase in their level of course engagement and in their perception of course content 
relevance compared to students in different sections of the same course taught primarily through lecture and 
discussion (TLD). Therefore, our research tested the following null hypotheses: 

1. There will be no significant difference in cognitive growth, as measured via pretest-posttest changes in 
postformal thinking skills, between students in PBL sections of a U.S. history survey course and 
students in different sections of the same course taught primarily through lecture and discussion. 

2. There will be no significant difference in self-reported postformal experience, as measured via an end 
of course questionnaire, between students in PBL sections of a U.S. history survey course and students 
in different sections of the same course taught primarily through lecture and discussion. 

3. There will be no significant difference in self-reported levels of course engagement, as measured via 
an end of study questionnaire, between students in PBL sections of a U.S. history survey course and 
students in different sections of the same course taught primarily through lecture and discussion. 

4. There will be no significant difference in self-reported level of course content relevance, as measured 
via and end of study questionnaire, between students in PBL sections of a U.S. history survey course 
and students in different sections of the same course taught tlirough primarily lecture and discussion. 

METHODS 

Three sections of the U.S. history survey course (America since 1890: HIST 2112) were taught by one of our 
research team members using our PBL method, and two sections of the same course were taught by another 
instructor using TLD. The three PBL sections included a total of 87 students (25 students in section one, 22 students 
in section two, and 40 students in section three). The two smaller PBL classes were first-year learning community 
sections, meaning that they were paired with a first-year seminal’ course. Students self-selected their section of HIST 
2112 during fall registration. HIST 2112 is a required general education course at our institution. The topical outline, 
activities, assessments, and dates of implementation v^ere identical in each of the three PBL sections, and included 
nine primary instructional topics or units, w ith six of the nine instructional units culminating in PBL activities. Each 
PBL activity took one or tw o 75-minute class periods to complete. A list of the nine units and the six PBL activities 
along with the URL to access the activities is provided in Appendix A. The other instructional methods used in the 
PBL-based course focused on providing context for the PBL activities and included lecture, discussion, and guided 
questions (Reisman & Wineburg, 2009). The entire course was designed to examine and analyze U.S. history 
through a progressive, conservative dialectal framew ork. Of the 87 PBL students, 78 agreed to participate in the 
study. 
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The control group included two sections of HIST 2112 with 112 students each, taught primarily by a traditional 
lecture and discussion (TLD) method wherein the instructor assigned students approximately a chapter a week from 
the course text, and provided PowerPoints that highlighted key content/concepts/ideas. The two TLD sections also 
included one reaction essay assignment based on one of three documentary films watched in class, all from the PBS 
American Experience series: The Civilian Conservation Corps (Samuals Sc Stone, 2009), Afy Lai (Samuals Sc 
Goodman. 2010), and The Triangle Fire (Samuals Sc Espar, 2011). The students were asked to address three 
questions in the essay : What was the film about? How r did it connect to the era being studied? How did it enhance, or 
not. your understanding of the era/topic? A connecting theme of the course was ''rhetoric vs. reality.” Of the 224 
students in the two TLD sections, 118 agreed to participate in the study. 

Measures 

Postformal Thought (PFT) Questionnaire. We administered the PFT (Sinnott Sc Johnson, 199?) to measure 
participants' levels of postformal thought. (See Cartwright, Galupo, Tyree, Sc Jennings, 2009 for discussion of 
reliability and validity .) The questionnaire included 10 statements that represent a different operation of postformal 
thinking. Participants responded to each statement by indicating the extent to which it characterized their own 
thinking (7 = very true. 1 = not true). The sum of the 10 items provided a summary PFT score for each participant. 
The pretest w r as administered during the first class of fall semester in the PBL sections, and during the second class 
in TLD sections. Posttests were administered on the last day of classes in each section. The PFT is included as 
Appendix C. 

End of Study Questionnaire (ESQ). We administered the ESQ to participants after all other data were collected. 
ESQ questions included a Likert rating of participants’ level of engagement in HIST 2112 (Question 1). a Likert 
rating on the level of relevance of course content and topics (Question 3) a question that prompted participants to 
reflect on the extent to w hich their experience in the HIST 2112 affected their ability 7 to think critically (Question 4). 
and one on the extent to which students believed that these skills would be useful beyond the course (Question 5). 
The ESQ is included as Appendix D. 

Methods of Analysis 

To test our PBL instructional model’s impact on the development of postformal thinking skills among U.S. history 
survey students, we conducted two analyses: one quantitative, and one incorporating mixed-methods. The 
quantitative analysis included a comparison of PFT mean-normalized gain scores of the PBL and TLD sections. The 
normalized gain score is “the ratio of the actual average gain (%<post> - %<pre>) to the maximum possible average 
gain (10G-%<pre>)” (Hake. 1998, p. 64). A one-way analysis of variance (ANOVA) w r as conducted to examine the 
holistic difference in postfonnal thinking between the tw r o groups. 

We used mixed methods to analyze responses to short-answer ESQ questions four and five. We used direct content 
analysis or deductive category application (Mayring, 2000; Potter Sc Levine-Donnerstein, 1999) to identify and define 
two operational definitions (see Appendix E) for two problem-solving systems: (a) closed systems-CS; and (b) 
postfonnal operational systems-PF. Two faculty researchers and two student researchers coded all responses 
individually. Student responses w : ere coded as self-reported PFT experience if the comment indicated the utilization 
of characteristics of relativistic and/or dialectical operations. For example, “I think my way of thinking has evolved in 
a way that I now r look at multiple solutions and sides to issues that I originally did not see?’ The third researcher 
examined the coded responses and identified categorized students as using postformal reasoning if three of the four 
coders (75%) agreed to that categorization. The third researcher then created a dichotomous nominal variable (1 = 
self-reported postformal-related experience in question four or five, 0 = no self-reported postfonnal-related 
experience in question four or five) in order to conduct an independent T-Test to examine the difference in self- 
reported experience with PFT. An independent T-Test w r as also conducted to examine the differences in self-reported 
levels of course engagement (ESQ Question 1) and self-reported levels of course content relevance (ESQ Question 
3) between the two groups. 
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RESULTS AND DISCUSSION 

Pretest PFT means of the PBL sections (n = 64) and TLD sections (n = 109) indicated that there was no significant 
difference in PFT between the two groups at the beginning of the study (PBL M = 50.77. SD = 7.61; TLD M = 
51.12, SD = 6.32), while the posttest means showed a greater mean PFT score for the PBL group (PBL M= 54.52, 
SD = 6.45; TLD M= 51.76. SD = 8.02). One-way AN OVA data for the normalized gain scores (Bao, 2006) showed 
a significantly greater net mean gain of 3.75 between pre- and posttest PFT scores in the PBL sections compared to a 
.64 mean gain in the TLD sections, F(L 171) = 8.84, p < .003, and a significantly greater normalized gain in the 
PBL sections (.079) than TLD sections (.025). F(l, 171) = 5.85, p < .017. We also analyzed PFT data to determine 
if there were mean gain differences between the smaller PBL learning community sections and the regular PBL 
section. There was no significant difference, with net gain means of 3.23 and 4.38 respectively. 

The significant results of the direct content analysis of ESQ questions four and five align well with the pre- and 
post-PFT results. Independent Mest data showed a significantly greater mean of self-reported postformal thinking 
experiences occurring in the PBL sections (n = 78. M = .87, SD = .34) compared to the TLD sections (n = 1 IS, M = 
.24, .SD = .48), r(193) = 10.95, p < .000. We thus rejected Hypothesis 2. Taken together, these results Indicate that 
our PBL instructional model yielded significantly greater postformal thinking scores compared to TLD; these results 
confirm those obtained by Wynn et al. (2014). 

We found the results of the analyses of comments from ESQ questions four and five compelling. The common 
themes noted from PBL student comments included their ability to recognize and better understand multiple truths 
or perspectives when confronted with a complex problem or issue, and the importance of utilizing multiple 
perspectives in the problem-solving and decision-making process. PBL students also recognized the applicability of 
postformal thinking systems beyond the confines of their U.S. history survey course. A sampling of PBL student 
comments reflecting these themes is presented in Appendix F. 

Of the 118 TLD students who provided comments, 28 comments were coded as indicating some level of experience 
with postformal thinking dynamics, specifically with recognition and utilization of multiple perspectives on 
historical events and the focus on “rhetoric vs. reality' 1 as a course theme. We hypothesized that the reaction essay 
assignment, and the perspectives introduced through the rhetoric vs. reality theme of the course, may have prompted 
the TLD student comments that indicated relativistic considerations. A sampling of these comments are also 
provided in Appendix F. 

Independent t -test data for responses to ESQ question one, which measured self-reported levels of course 
engagement, show ed PBL students reported significantly greater levels of engagement (n = 78, M= 4.25, SD = .73) 
compared to TLD students (n = 116, M = 3.29. SD = 1.06), r( 191) = 6.87, p < .000. Therefore, we rejected 
Hypothesis 3. 

The similar engagement results from Wynn et al. (2014) confirm our conclusion that the primary difference in level 
of engagement between the two groups may be explained by the explicit learner-centered, social learning dynamic 
that is facilitated by our PBL model in comparison to the more teacher-centered learning dynamic of the TLD 
sections. Our comparison of PBL and TLD student comments offered a compelling glimpse into how the two groups 
differed in their perceptions of what constitutes course engagement in general. We selected and reviewed comments 
from students in both groups who scored their levels of engagement as fours or fives. TLD students tended to frame 
explanations of engagement within a traditional teacher-centered learning dynamic compared to the more learner- 
centered, active explanations offered by PBL students. For example, PBL student #5 stated, “I was actively forming 
opinions on subjects through information presented by classmates as well as my instructor, unlike previous courses I 
have taken where all information presented comes from a singular source, the teacher. 1 ' TLD student #136 stated, “I 
came to almost every class this semester except maybe two. I take notes and have done well on the tests. If I w asn’t 
engaged, I would not be doing well.” A comparative sampling of PBL and TLD student comments from ESQ 
question one is provided in Appendix F. 

Independent t -test data for responses to ESQ question three, which measured self-reported perceptions of content 
relevance, again showed PBL students reporting significantly higher ratings (n = 78, M= 4.77, SD = .51) than TLD 
students (n = 116, M= 4.13, SD = 1.24), ^(191) = 4.24,/? < .000. This led us to reject Hypothesis 4. 
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The PEL course was designed to guide students not only to connect the past to the present, but also to apply the 
advanced problem-solving skills gained through confronting contextualized historical problems to current issues, 
including The Affordable Care Act: Immigration Reform; Debt, Spending. Taxes, and a Balanced Budget 
Amendment; Climate Change Cap and Trade; and Minimum Wage. In general. PEL student comments indicated 
that their high ranking of content relevance was due to their recognition of this dynamic. For example, PEL student 
#19 stated. “In this class we have dealt with immigration, civil rights, and even ISIS. It [content] w r as very relevant 
because we dealt with these topics using the same lens as we did when dealing with historical events.” 

CONCLUSIONS AND RECOMMENDATIONS 

Results from the current study confirm the findings of Wynn et al. (2014). Our PEL instructional method, which 
includes metaeognitive reflection, has a significant relationship with the development of postformal thinking skills 
compared to a TLD method in a U.S. history survey course context. We believe that several factors may explain 
these results. Basseches (2005) and Sinnott (1998) argued that individuals must be confronted by the diverse 
perspectives, multiple truths, and contradictions inherent in complex problems and issues in order to first recognize 
the Inadequacy of formal/absolutist thinking, and second, to begin seeking more adequate/advanced thinking 
systems. Our PEL model was designed to prompt metaeognitive reflection in a way that allows instructors to guide 
students toward the recognition and practice of multiple thinking systems and to judge the extent to which these 
systems are adequate for successfully addressing complex problems and issues. We facilitated these processes 
through an initial thinking systems tutorial at the end of the first PEL activity, followed by the administration of the 
metaeognitive reflection questionnaire. W 7 e also administered the metaeognitive questionnaire after the second PEL 
activity. PBL students began recognizing the dynamics of intuitive, analytical, relativistic, and dialectical thinking, 
as well as domain-specific thinking systems, like historical thinking, very early in the semester and they learned 
relatively quickly how r to judge the extent to w hich these systems were usefully practiced. Most PBL students w ere 
able to initiate metaeognitive reflection without the use of the questionnaire after the third PBL activity. We believe 
this working knowledge of the multiple thinking systems practiced in advanced problem-solving helps explain the 
greater postformal thinking gains among PBL student. 

We recognize that U.S. history survey course instructors, and survey course instructors in general, might be hesitant 
to take on the task of guiding the metaeognitive reflection process. It takes instructional time and some working 
knowledge of thinking systems. The metaeognitive reflection questionnaire provided in Appendix B is offered to 
introduce the thinking systems involved in problem solving and decision making and to facilitate reflective 
discussions. Instructors do not have to be experts to guide the process. 

Two other factors w r e believe are significant in explaining the holistic differences between the PEL and TLD 
sections involve class size and content coverage. The social learning dynamics that are so critical to the success of 
PBL activities are more easily facilitated in smaller classes (Wynn, 2010, 2015; Wynn et al., 2014). TLD student 
#163 framed the impact of class size on engagement in the following comment on ESQ question one: “It is hard to 
be fully engaged in a course that covers so much material and has such a large class size. We are only given the 
ability 7 to take notes and give a few r responses.” We concur with this observation and recommend that the 
chronological coverage approach be significantly adapted to accommodate a workable number of activities if 
instructors want to implement PEL. Each instructor should select “turning-point” problems and issues on which to 
develop PEL activities, and should determine the number of activities to include, recognizing that confronting 
students with multiple problems or issues may be necessary to facilitate more adequate and sustainable thinking 
skills. We recommend a maximum 40-student class size for PBL sections. 

We also recommend that current issues be included as the basis of PBL activities during the last phase of a PBL- 
based U.S. History II survey course; students may apply the postformal problem-solving skills to current issues in 
the manner that they practiced with historical issues. PBL student comments on ESQ question three clearly indicated 
that students attributed content relevance, in part, to their engagement wdth contemporary problems and issues that 
may directly affect their lives. This focus also aligns well w ith the AAC&U (2015) general education principle cited 
earlier regarding students gaining proficiencies “tlirough problem-centered work on significant issues relevant to 
their interests and aims” (p. 3). Confronting current issues may also help students recognize the applicability' of 
postfomal thinking systems and even historical inquiiy, bey-ond the confines of their U.S. history survey- courses. 
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LIMITATIONS AND CHALLENGES 

Several limitations of the current study should be noted. First, two different instructors taught the PEL and TDL 
sections, possibly leading to the occurrence of uncontrolled extraneous events. Second, the relatively small sample 
size in both the PEL and TLD sections may limit the generalizability and utility of the findings. Third, as stated 
previously, class size is certainly a factor in successfully implementing PEL activities. The large class size in both 
TLD sections could affect the success of TLD instructors facilitating more advanced thinking skills, engagement, 
and perceptions of content relevance among students. Studies measuring outcomes in PEL and TLD sections of 
similar size should be conducted. We also note the fact that the two smaller PEL sections were affiliated with 
learning communities, which may be a confounding variable. 

A significant challenge in implementing a PBL-based U.S. history survey is the amount of time and effort necessary 
to construct the PEL activities. The researcher who taught the PEL sections spent several months developing the 
activities and gathering the necessary resources and primary documents to support the activities and related 
assignments. With that noted, we consider the social learning environment and the resulting levels of student 
engagement, content relevance, and advanced thinking skills that are facilitated through our PEL instructional model 
to be well worth the effort. 

We encourage other SoTL scholars in history 7 and other disciplines to test the impact of PEL arid metacognitive 
reflection on their survey course students. As stated earlier, AAC&U (2015) is calling for problem-based college 
curricula to guide students toward the development of useful and sustainable problem-solving skills. We believe 
general education curricula should provide meaningful and relevant experiences that promote demonstrable and 
sustainable proficiency in post formal thinking. We offer our PEL instructional model as an option that survey course 
instructors may use to facilitate these skills. In this time of intense political polarization, and given the ever- 
increasing issues that affect the well-being and security of all of us, such cognitive and deliberative capacities are 
sorely needed. 
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HIST 2112 PEL Instructional Units 

Unit 1 - Political Dynamics of the 1890s 

Unit 2 - The U.S. as an Empire: Global Power Structure (1890-1905) 

PEL Activity - The Question of US. Expansion: Expansionists versus A nti - Expans i o rusts 

Unit 3 — Social and Political Dynamics in the Progressive Era 

Unit 4 - The Nation at War 

PEL Activity - Wilson and the Paris Peace Conference: Constructing the Treaty of Versailles - The full 
activity, including detailed procedures, materials, assessments, rubrics, and Power Point slides may be 
accessed at http://tinyur 1.com/wynnpbl2, password: PBL2. 

Unit 5 - Economic Expansion of the 1920s, The Depression, Franklin D. Roosevelt and the New Deal 
PEL Activity - Solving the Problems of the Depression: Constructing the New Deal 

Unit 6 - America and the World (1921-1945) 

PEL Activity - The Atomic Bomb: Truman ? s Decision and Its Impact 

The Post War Era and Beyond - 1945 to Present 

Unit 7 - The Cold War and Beyond (includes current foreign policy challenges) 

Unit 8 — Civil Rights in the U.S.: Tracing Social, Economic, and Political Dynamics in the Last Half of the 20 th 
Century PEL Activity — The Issue of Affirmative Action: The Atlanta Case 

Each of the above activities may be accessed at 

http s://www. taskstre am. c om/ts/wynn3 /HIS T2112 Probl einB a sedActivities, p assword: Wynn 1. 

Unit 9 - Challenges of the New Century 

PEL Activity - Student Decisions on Current Issues: The Affordable Care Act; Immigration Reform; Debt , 
Spending, Taxes , and a Balanced Budget Amendment; Climate Change and Cap & Trade; Minimum Wage. 
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Metacognitive Reflection Questionnaire (Adapted from Wynn et al.. 2014) 

Please respond to each statement below by circling the number that best describes the thinking/reasoning you 
used during this activity . 

1 = Never (N) 2 = Rarely (R) 3 = Occasionally (S) 

4 = Somewhat Often (SO) 5 = Often (O) 6 = Very Often (VO) 

N R S SO O VO 

1. I used intuitive or emotional thinking (It felt right.) as I made a decision on this problem/issue. (Intuitive 
Thinking) 

1 2 3 4 5 6 

2. I used logical-analytical thinking (application of logical operations) to develop correct solution as I made a 
decision on this problem/is sue. (Formal Thinking) 

1 2 3 4 5 6 

3. I considered context, multiple causes, multiple points of view, and conflicting ideas about what is true and 
relevant related to this problem/issue as I reached a conclusion or decision. (Postformal Thinking: 
Relativistic) 

1 2 3 4 5 6 

4. I searched for and used inconsistencies and contradictions inherent in this problem/issue, sought to 
understand why those contradictory perspectives exist, and sought to resolve those contradictions as part of 
the problem-solving process. (Postformal Thinking: Relativistic/Dialectical) 

1 2 3 4 5 6 

5. I recognized that often there is no '‘correct” answer when dealing with complex problems/issues like this 
one. (Postformal Thinking: Relatistic/Dialectical) 

1 2 3 4 5 6 

6. I considered how change could affect this problem'issue and possibly my opinion;decision. 
(Postformal Thinking: Dialectical) 

1 2 3 4 5 6 

Please respond to the following question on the back of this questionnaire. 

7. Describe the various thinking systems you utilized during this problem-based activity, (from those listed 
above, and from more discipline specific processes like historical thinking, mathematical 
computation/estimation, etc.) How significant were these processes in helping you solve the problem or 
make a decision? 
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Complex Postfonnal Thought (PFT) Questionnaire (Sinnott & Johnson, 1997) 

The Complex Postfonnal Thought Questionnaire is reprinted with permission from Springer Publishing Company. 
Please respond to each item below by circling the number that best describes von on the following scale: 

1 = Not True (of self) and 7 = Very True (of self). 

1. I see the paradoxes in life (Paradoxes are inherent contradictions in reality.) 

1 2 3 4 5 6 7 

2. I see more than one method that can be used to reach a solution or decision on a problem or issue. 

1 2 3 4 5 6 7 

3. I am aware that I can decide which reality or truth to experience at a particular time; but I know 7 that reality 
and truth is really multi-level and more complicated. 

1 2 3 4 5 6 7 

4. There are many “right” ways to define any life experience; I must make a final decision on how I define 
the problems of life. 

1 2 3 4 5 6 7 

5. I am aware that sometimes “succeeding” in the everyday world means finding a concrete answer to one of 
life's problems; but sometimes it means finding a correct path that w r ould cany' me through any problems of 
this type. 

1 2 3 4 5 6 7 

6. Almost all problems can be solved by logic, but this may require different types of e logics.” 

1 2 3 4 5 6 7 

7. I tend to see several causes connected with any event. 

1 2 3 4 5 6 7 

8. I see that a given dilemma always has several good solutions. 

1 2 3 4 5 6 7 

9. I realize that I often have several goals in mind, or that life seems to have several goals in mind for me. So I 
go toward more than one in following my path in life. 

1 2 3 4 5 6 7 

10. I can see the hidden logic in others 1 solutions to the problem of life, even if I don’t agree with their 
solutions and follow 7 mv ow n path. 

1 2 3 4 5 6 7 
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End of Study Questions (Wynn et al. ? 2014) 

Title of the Research Study: The effect of problem-based learning and meta cognitive reflection on the 
development of postformal thinking among U.S. history survey students 

Thank you for participating in this study. Please answer the 5 questions below based on your experience this semester 
in HIST 2112. 

1. Rank your level of engagement (active participation) in this course. 

(1 = not engaged, 5 = fully engaged). 

1 2 3 4 5 

Explain your response. 


2. How does your ranking of engagement in this course compare to other history/psychology courses you’ve 

taken (in college or high school)? 

Explain your response. 


3. Rank the level of relevancy of the content of this course. How r relevant were the topics (content areas)? 

(1 = irrelevant, 5 = very relevant). 

Explain your response. 


4. Do you believe you have expanded your ability 7 to think critically as a result of this course? If so, can you 

explain the how your thinking has changed and or evolved? 


5. To w hat extent do you believe you may utilize the thinking skills you may have gained in this course as 

you continue your education and life in general? 

Explain your response. 
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Operational Definitions: Closed Systems (CS) and Postformal Operational (PF) (Wynn et aL 2014) 

CS can be identified by the following characteristics: 

• Focus on a limited number of aspects to the exclusion of other potentially useful aspects 

• Use of a familiar problem solving framework 

• Expectation of a single right answer applicable to all similar circumstance 

Analytical/Fomial (Closed Systems) Problem-Solving - A closed systems problem-solver will generally apply a 
practiced systematic/formal problem-solving framework based on previous experience with similar problems to 
solve the problem at hand. This framework involves a problem-solving dynamic based on a limited number of 
variables, w ith other important aspects of the problem often judged as irrelevant to the solution. This causes formal 
thinkers to expect to produce a single right answer that will apply to all similar circumstances. 

People are using PF when they recognize some or all of the following: 

• What is considered to be true can change when perspective and context change 

• Contradictions are often critical to understanding the complexities of a problem 

• The sides surrounding contradictions are interrelated and must be accommodated in the development 
of resolution alternatives 

• Some problems and issues don’t have simple and neat solutions 

• Even though a solution to a messy, real-life problem is identified, there will always be new challenges 
as the world and the people involved change. 

Postformal Problem-Solving - Relativistic thinkers recognize that as a person’s perspective or context changes, so 
too does her/his perspective on what is true. Rather than seeing problems and issues through the lens of fixed truths, 
and/or good vs. bad. relativistic thinkers recognize that context and contradictions are key to understanding the 
complexities of a problem, developing potential resolution alternatives, and recognizing the fact that no resolution 
may be possible for some problems and issues. 

Dialectical thinking involves the integration of relativistic thinking with the recognition that both sides of 
contradictions within a problem/issue are interrelated and connected and are critical in the development of resolution 
alternatives. Inconsistencies and contradictions within problems and issues become catalysts in the application of 
multiple cognitive systems as dialectical thinkers seek resolutions that lead to higher levels of understanding and 
cognition. Dialectical thinkers also recognize that any resolution or stability that may result from dialectical 
problem-solving operations will be perpetually challenged by new challenges, changes, and a potential tension-to 
resolution-to tension cycle dynamic. 
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Sampling of Student Comments from ESQ Questions 

Sample PEL Student Comments from ESQ Questions Four or Five (PET Experience): 

• Student #19-PBL Section 1 - “I personally have never really been a person to pick sides and this class 
only strengthened that. I was reminded that both sides of an argument may have very valid points. One 
other way that I feel like I have gotten better is collaborating with others to make a better solution. It is 
silly to debate about something and not come up with a solution. I learned how to reach a solution with 
people who have very different viewpoints than me.” 

• Student #20-PBL Section 1 - “Yes. I can look at more than one view point. I can look beyond gut 
thinking. I can understand why opposing views think the way they do and use their perspective in my 
thinking toward a solution. I will use this in all aspects of my life. It is a great thing to have and 
utilize.” 

• Student #3 5-PBL Section 2 - “Yes. I can now look at both sides before I make a decision. Both sides 
have a point and both points need to be viewed before making a decision. I believe I will use this for 
the rest of my life. I learned to stop and think before doing anything and to look into both views.” 

• Student #39-PBL Section 2 — “Yes. Before, if I had an opinion I would only look at it from my point of 
view and I would automatically think I believe the other side is wrong. This class made me look at 
both sides critically and then come up with a decision. 

• Student #71-PBL Section 3 - “Yes. I believe this course has allowed me to expand my thinking skills 
very much. As a result of this course. I have understood that there are several possible solutions to 
complicated problems. Moreover, it has taught me to think critically about problems and understand 
multiple sides.” 

• Student #85-PBL Section 3 - “Yes. [The instructor] made us realize that here are always more points 
of view other than our own. For example, the activity about the end of WWI and how to punish 
Germany showed me Germany's side for the first time ever. Our class was able to find compromise in 
every situation we were given. It makes me wonder how our government fails to do the same.” 

Sample TLD Student Comments from ESQ Questions Four or Five fPFT Experience): 

• Student #105-TDL Section 1 - “I have expanded my thinking because I see and understand different 
reasons for why things happened the way they did, and how some things could have been prevented. I 
can listen to and respect others' opinions even if I disagree.” 

• Student +/152-TDL Section 1 - “I utilized some of the perspectives focused on in this course to try to 
see problems from different points of view.” 

• Student #259-TDL Section 2 - “I think so. Fve learned there’s hidden sides to a lot of different things 
and situations” 

• Student#336-TDL Section 2 - “Yes. Since one of the main course themes was rhetoric vs. reality, it 
makes me question more things in our society and try to dive deeper into where these problems 
originated.” 

Sample PBL Student Comments from ESQ Question One (Engagement): 

• Student #5-PBL Section I - “I was actively forming opinions on subjects through information 
presented by classmates as well as my instructor, unlike previous courses I have taken where all 
information presented comes from a singular source, the teacher.” 

• Student #6-PBL Section 1 - “When we did activities involving what we would do in certain situations, 
I was fully engaged. I was also fully engaged when learning about current issues.” 

• Student #33-PBL Section 2 - “I always felt like I was fully involved in the course, not only because of 
the way we had to come up with solutions after analyzing the situations/problems, but also because of 
the way the lecture was presented.” 


Copyright by author(s); CC-BY 


18 


The Clute Institute 






Journal of College Teaching & Learning - First Quarter 2016 


Volume 13, Number 1 


• Student #34-PBL Section 2 - “This class was constantly challenging my beliefs and had me actively 
thinking about why I believe what I believe. The class was very engaging. 

• Student #64-PBL Section 3 — “Five. Professor has us analyze situations and asks us to come up with 
solutions that we share in class.” 

• Student #78-PBL Section 3 - “Always on the edge of my seat in this class - love hearing both sides of 
any given point.” 


Sample TLD Student Comments from ESQ Question One (Engagement): 

• Student #136-TLD Section 1 - “I came to almost every class this semester except maybe two. I take 
notes and have done well on the tests. If I wasn't engaged, I would not be doing well.” 

• Student #178-TLD Section 1 — “I feel like I have been very engaged because I have only missed one 
class due to unforeseen circumstances, and pay attention;take notes every class period.” 

• Student #225-TLD Section 2 - “I feel engaged with the material I am learning because [my professor] 
has taught me the material from different angles and perspectives not known to me. Another thing is 
that lie is very knowledgeable and is an intellectual man in nature.” 

• Student #291-TLD Section 2 - “I attended all but two class sessions, read the textbook for roughly half 
of the semester. I paid attention to lectures and took notes, but didn’t' often raise my hand to answer 
questions because of shyness or not know ing the answ er to a question.” 
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